{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Numpy 进阶\n",
    "\n",
    "在本附录中，我将更深入地研究用于数组计算的NumPy库。这将包括有关ndarray类型的更多内部细节以及更高级的数组操作和算法。\n",
    "\n",
    "本附录包含其他主题，不一定需要线性阅读。\n",
    "\n",
    "## A.1 ndarray对象内部\n",
    "\n",
    "NumPy ndarray提供了一种手段，可以将同构数据块（连续或跨步）解释为多维数组对象。数据类型（或dtype）确定如何将数据解释为浮点型，整数型，布尔型或我们一直关注的任何其他类型。\n",
    "\n",
    "使ndarray灵活的部分原因在于，每个数组对象都是数据块上的跨步视图。例如，您可能想知道数组视图arr [:: 2，::-1]如何不复制任何数据。 原因是ndarray不仅仅是一块内存和一个dtype；它还具有“跨步”信息，使阵列能够以不同的步长在内存中移动。更准确地说，ndarray内部包含以下内容：\n",
    "\n",
    "- 指向数据的指针，即RAM或内存映射文件中的数据块\n",
    "- 数据类型或dtype，描述数组中固定大小的值单元格\n",
    "- 表示数组形状的元组\n",
    "- 跨度的元组，整数表示要沿维度前进一个元素的要“步进”的字节数\n",
    "\n",
    "有关ndarray内部模型的简单模型，请参见图A-1。\n",
    "\n",
    "![app1](images/app.1.png)\n",
    "\n",
    "例如，一个10×5的数组将具有形状（10，5）："
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "(10, 5)"
      ]
     },
     "execution_count": 1,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "import numpy as np\n",
    "\n",
    "np.ones((10, 5)).shape"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "一个典型的（C阶）3x4×5数组float64（8字节）值具有步幅（160、40、8）（了解这些步幅可能很有用，因为通常，特定轴上的步幅较大 ，沿该轴执行计算的成本更高）："
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "(160, 40, 8)"
      ]
     },
     "execution_count": 2,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "np.ones((3, 4, 5), dtype=np.float64).strides"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "虽然很少有典型的NumPy用户对数组的跨步感兴趣，但它们是构造“零复制”数组视图的关键要素。跨度甚至可以是负数，从而使数组可以在内存中“向后移动”（例如，在obj [::-1]或obj [:, ::-1]这样的切片中就是这种情况）。\n",
    "\n",
    "### NumPy dtype层次结构\n",
    "\n",
    "您可能偶尔会有需要检查数组是否包含整数，浮点数，字符串或Python对象的代码。因为有多种类型的浮点数（float16到float128），所以检查dtype是否在类型列表中会非常冗长。幸运的是，dtypes具有超类，例如np.integer和np.floating，它们可以与np.issubdtype函数结合使用："
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {},
   "outputs": [],
   "source": [
    "ints = np.ones(10, dtype=np.uint64)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {},
   "outputs": [],
   "source": [
    "floats = np.ones(10, dtype=np.float32)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "True"
      ]
     },
     "execution_count": 5,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "np.issubdtype(ints.dtype, np.integer)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "True"
      ]
     },
     "execution_count": 6,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "np.issubdtype(floats.dtype, np.floating)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "您可以通过调用类型的mro方法查看特定dtype的所有父类："
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "[numpy.float64,\n",
       " numpy.floating,\n",
       " numpy.inexact,\n",
       " numpy.number,\n",
       " numpy.generic,\n",
       " float,\n",
       " object]"
      ]
     },
     "execution_count": 7,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "np.float64.mro()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "因此，我们还有："
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "True"
      ]
     },
     "execution_count": 8,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "np.issubdtype(ints.dtype, np.number)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "大多数NumPy用户永远都不必知道这一点，但它有时会派上用场。有关dtype层次结构和父子类关系的图表，请参见图A-2。\n",
    "\n",
    "![app2](images/app.2.png)\n",
    "\n",
    "## A.2 矩阵操作进阶\n",
    "\n",
    "除了花式的索引，切片和布尔子集以外，还有许多方法可以使用数组。尽管数据分析应用程序的许多繁重工作都是由熊猫中的高级功能来处理的，但您有时可能需要编写一种在现有库中找不到的数据算法。\n",
    "\n",
    "### 调整数据形状\n",
    "\n",
    "在许多情况下，您可以将数组从一种形状转换为另一种形状，而无需复制任何数据。为此，将表示新形状的元组传递给reshape数组实例方法。例如，假设我们有一个一维值数组，希望将其重新排列成矩阵（结果如图A-3所示）：\n",
    "\n",
    "![app4](images/app.3.png)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "array([0, 1, 2, 3, 4, 5, 6, 7])"
      ]
     },
     "execution_count": 9,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "arr = np.arange(8)\n",
    "\n",
    "arr"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "array([[0, 1],\n",
       "       [2, 3],\n",
       "       [4, 5],\n",
       "       [6, 7]])"
      ]
     },
     "execution_count": 10,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "arr.reshape((4, 2))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "多维数组也可以调整："
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 11,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "array([[0, 1, 2, 3],\n",
       "       [4, 5, 6, 7]])"
      ]
     },
     "execution_count": 11,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "arr.reshape((4, 2)).reshape((2, 4))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "传递的形状尺寸之一可以是–1，在这种情况下，将从数据推断用于该尺寸的值："
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 12,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "array([[ 0,  1,  2],\n",
       "       [ 3,  4,  5],\n",
       "       [ 6,  7,  8],\n",
       "       [ 9, 10, 11],\n",
       "       [12, 13, 14]])"
      ]
     },
     "execution_count": 12,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "arr = np.arange(15)\n",
    "\n",
    "arr.reshape((5, -1))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "由于数组的shape属性是一个元组，因此也可以将其传递给整形："
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 13,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "(3, 5)"
      ]
     },
     "execution_count": 13,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "other_arr = np.ones((3, 5))\n",
    "other_arr.shape"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 14,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "array([[ 0,  1,  2,  3,  4],\n",
       "       [ 5,  6,  7,  8,  9],\n",
       "       [10, 11, 12, 13, 14]])"
      ]
     },
     "execution_count": 14,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "arr.reshape(other_arr.shape)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "从一维到高维的相反整形操作通常称为展平或拉平："
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 15,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "array([[ 0,  1,  2],\n",
       "       [ 3,  4,  5],\n",
       "       [ 6,  7,  8],\n",
       "       [ 9, 10, 11],\n",
       "       [12, 13, 14]])"
      ]
     },
     "execution_count": 15,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "arr = np.arange(15).reshape((5, 3))\n",
    "\n",
    "arr"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 16,
   "metadata": {
    "scrolled": true
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "array([ 0,  1,  2,  3,  4,  5,  6,  7,  8,  9, 10, 11, 12, 13, 14])"
      ]
     },
     "execution_count": 16,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "arr.ravel()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "如果结果中的值在原始数组中是连续的，那么ravel不会生成基础值的副本。flatten方法的行为类似于ravel，但它始终返回数据的副本："
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 17,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "array([ 0,  1,  2,  3,  4,  5,  6,  7,  8,  9, 10, 11, 12, 13, 14])"
      ]
     },
     "execution_count": 17,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "arr.flatten()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "数据可以按不同顺序重塑或整理。对于新的NumPy用户而言，这是一个微妙的主题，因此是下一个子主题。\n",
    "\n",
    "### C与Fortran顺序\n",
    "\n",
    "NumPy使您可以控制和灵活地控制数据在内存中的布局。默认情况下，按行主要顺序创建NumPy数组。从空间上讲，这意味着如果您有一个二维数据数组，则该数组每一行中的项目都存储在相邻的内存位置中。行主要顺序的替代方法是列主要顺序，这意味着每列数据中的值都存储在相邻的存储位置中。\n",
    "\n",
    "由于历史原因，行和列的主要顺序也分别称为C和Fortran顺序。在FORTRAN 77语言中，矩阵都是主列。\n",
    "\n",
    "诸如reshape和ravel之类的函数接受一个order参数，该参数指示使用数组中数据的顺序。在大多数情况下，通常将其设置为“ C”或“ F”（也有一些不常用的选项“ A”和“ K”；请参见NumPy文档，并参考图A-3以获得这些选项的说明） ）："
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 18,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "array([[ 0,  1,  2,  3],\n",
       "       [ 4,  5,  6,  7],\n",
       "       [ 8,  9, 10, 11]])"
      ]
     },
     "execution_count": 18,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "arr = np.arange(12).reshape((3, 4))\n",
    "\n",
    "arr"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 19,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "array([ 0,  1,  2,  3,  4,  5,  6,  7,  8,  9, 10, 11])"
      ]
     },
     "execution_count": 19,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "arr.ravel()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 20,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "array([ 0,  4,  8,  1,  5,  9,  2,  6, 10,  3,  7, 11])"
      ]
     },
     "execution_count": 20,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "arr.ravel('F')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "重塑具有两个以上维度的数组可能会有些弯腰（请参见图A-3）。C和Fortran顺序之间的主要区别在于维度的行走方式：\n",
    "\n",
    "> C/行主要顺序\n",
    "    首先移动较大的尺寸（例如，在轴0前进之前先在轴1上移动）。\n",
    "> Fortran/列主要顺序\n",
    "    最后移动较大的尺寸（例如，沿轴1前进之前的轴0）。\n",
    "    \n",
    "### 连接和拆分数组\n",
    "\n",
    "numpy.concatenate接受数组的序列（元组，列表等），并沿输入轴按顺序将它们连接在一起："
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 21,
   "metadata": {},
   "outputs": [],
   "source": [
    "arr1 = np.array([[1, 2, 3], [4, 5, 6]])\n",
    "arr2 = np.array([[7, 8, 9], [10, 11, 12]])"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 22,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "array([[ 1,  2,  3],\n",
       "       [ 4,  5,  6],\n",
       "       [ 7,  8,  9],\n",
       "       [10, 11, 12]])"
      ]
     },
     "execution_count": 22,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "np.concatenate([arr1, arr2], axis=0)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 23,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "array([[ 1,  2,  3,  7,  8,  9],\n",
       "       [ 4,  5,  6, 10, 11, 12]])"
      ]
     },
     "execution_count": 23,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "np.concatenate([arr1, arr2], axis=1)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "有一些方便的功能，例如vstack和hstack，用于常见的串联。前面的操作可以表示为："
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 24,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "array([[ 1,  2,  3],\n",
       "       [ 4,  5,  6],\n",
       "       [ 7,  8,  9],\n",
       "       [10, 11, 12]])"
      ]
     },
     "execution_count": 24,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "np.vstack((arr1, arr2))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 25,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "array([[ 1,  2,  3,  7,  8,  9],\n",
       "       [ 4,  5,  6, 10, 11, 12]])"
      ]
     },
     "execution_count": 25,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "np.hstack((arr1, arr2))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "split, 另一方面，将一个数组沿一个轴切成多个数组："
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 26,
   "metadata": {},
   "outputs": [],
   "source": [
    "arr = np.random.randn(5, 2)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 27,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "array([[-1.61901755, -1.9332683 ],\n",
       "       [ 0.25073952, -1.03502051],\n",
       "       [-0.33103284,  0.61901676],\n",
       "       [ 0.44292818,  0.85685508],\n",
       "       [-0.94375608, -0.63382524]])"
      ]
     },
     "execution_count": 27,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "arr"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 28,
   "metadata": {},
   "outputs": [],
   "source": [
    "first, second, third = np.split(arr, [1, 3])"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 29,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "array([[-1.61901755, -1.9332683 ]])"
      ]
     },
     "execution_count": 29,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "first"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 30,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "array([[ 0.25073952, -1.03502051],\n",
       "       [-0.33103284,  0.61901676]])"
      ]
     },
     "execution_count": 30,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "second"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 31,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "array([[ 0.44292818,  0.85685508],\n",
       "       [-0.94375608, -0.63382524]])"
      ]
     },
     "execution_count": 31,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "third"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "传递给np.split的[1，3]值指示将数组拆分为多个片段的索引。"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "有关所有相关级联和拆分功能的列表，请参见表A-1，其中某些功能仅出于非常通用的级联的便利而提供。\n",
    "\n",
    "#### Stacking 帮助: r_ and c_\n",
    "\n",
    "NumPy命名空间中有两个特殊对象r_和c_，它们使堆栈数组更简洁："
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 32,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "array([[ 0.        ,  1.        ],\n",
       "       [ 2.        ,  3.        ],\n",
       "       [ 4.        ,  5.        ],\n",
       "       [ 0.2147942 ,  1.97260327],\n",
       "       [ 0.39373445, -0.52875549],\n",
       "       [-0.17477746,  1.27464435]])"
      ]
     },
     "execution_count": 32,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "arr = np.arange(6)\n",
    "\n",
    "arr1 = arr.reshape((3, 2))\n",
    "arr2 = np.random.randn(3, 2)\n",
    "\n",
    "np.r_[arr1, arr2]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 33,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "array([[ 0.        ,  1.        ,  0.        ],\n",
       "       [ 2.        ,  3.        ,  1.        ],\n",
       "       [ 4.        ,  5.        ,  2.        ],\n",
       "       [ 0.2147942 ,  1.97260327,  3.        ],\n",
       "       [ 0.39373445, -0.52875549,  4.        ],\n",
       "       [-0.17477746,  1.27464435,  5.        ]])"
      ]
     },
     "execution_count": 33,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "np.c_[np.r_[arr1, arr2], arr]"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "这些还可以将切片转换为数组："
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 34,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "array([[  1, -10],\n",
       "       [  2,  -9],\n",
       "       [  3,  -8],\n",
       "       [  4,  -7],\n",
       "       [  5,  -6]])"
      ]
     },
     "execution_count": 34,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "np.c_[1:6, -10:-5]"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "有关使用c_和r_可以执行的操作的更多信息，请参见文档字符串。\n",
    "\n",
    "### 重复元素：平铺并重复\n",
    "\n",
    "用于重复或复制数组以生成更大数组的两个有用的工具是repeat和tile功能。repeat多次复制数组中的每个元素，从而产生更大的数组："
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 35,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "array([0, 1, 2])"
      ]
     },
     "execution_count": 35,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "arr = np.arange(3)\n",
    "\n",
    "arr"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 36,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "array([0, 0, 0, 1, 1, 1, 2, 2, 2])"
      ]
     },
     "execution_count": 36,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "arr.repeat(3)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "> 与其他数组编程框架（如MATLAB）相比，NumPy很少需要复制或重复数组。原因之一是广播通常可以更好地满足这一需求，这是下一部分的主题。\n",
    "\n",
    "默认情况下，如果传递整数，则每个元素将重复该次数。如果传递整数数组，则每个元素可以重复不同的次数："
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 37,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "array([0, 0, 1, 1, 1, 2, 2, 2, 2])"
      ]
     },
     "execution_count": 37,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "arr.repeat([2, 3, 4])"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "多维数组可以使其元素沿特定轴重复。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 38,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "array([[ 0.44663835, -1.20995865],\n",
       "       [ 0.66875485,  0.59349705]])"
      ]
     },
     "execution_count": 38,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "arr = np.random.randn(2, 2)\n",
    "\n",
    "arr"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 39,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "array([[ 0.44663835, -1.20995865],\n",
       "       [ 0.44663835, -1.20995865],\n",
       "       [ 0.66875485,  0.59349705],\n",
       "       [ 0.66875485,  0.59349705]])"
      ]
     },
     "execution_count": 39,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "arr.repeat(2, axis=0)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 40,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "array([[ 0.44663835,  0.44663835, -1.20995865, -1.20995865],\n",
       "       [ 0.66875485,  0.66875485,  0.59349705,  0.59349705]])"
      ]
     },
     "execution_count": 40,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "arr.repeat(2, axis=1)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "请注意，如果没有传递轴，则数组将首先被展平，这可能不是您想要的。类似地，当重复多维数组以给定切片重复不同次数时，可以传递整数数组："
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 41,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "array([[ 0.44663835, -1.20995865],\n",
       "       [ 0.44663835, -1.20995865],\n",
       "       [ 0.66875485,  0.59349705],\n",
       "       [ 0.66875485,  0.59349705],\n",
       "       [ 0.66875485,  0.59349705]])"
      ]
     },
     "execution_count": 41,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "arr.repeat([2, 3], axis=0)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 42,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "array([[ 0.44663835,  0.44663835, -1.20995865, -1.20995865, -1.20995865],\n",
       "       [ 0.66875485,  0.66875485,  0.59349705,  0.59349705,  0.59349705]])"
      ]
     },
     "execution_count": 42,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "arr.repeat([2, 3], axis=1)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "另一方面，tile是沿轴堆叠数组副本的快捷方式。在视觉上，您可以认为它类似于“铺设瓷砖”："
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 43,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "array([[ 0.44663835, -1.20995865],\n",
       "       [ 0.66875485,  0.59349705]])"
      ]
     },
     "execution_count": 43,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "arr"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 44,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "array([[ 0.44663835, -1.20995865,  0.44663835, -1.20995865],\n",
       "       [ 0.66875485,  0.59349705,  0.66875485,  0.59349705]])"
      ]
     },
     "execution_count": 44,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "np.tile(arr, 2)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "第二个参数是瓦片的数量；使用标量时，平铺会逐行进行，而不是逐列进行。tile的第二个参数可以是表示“平铺”布局的元组："
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 45,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "array([[ 0.44663835, -1.20995865],\n",
       "       [ 0.66875485,  0.59349705]])"
      ]
     },
     "execution_count": 45,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "arr"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 46,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "array([[ 0.44663835, -1.20995865],\n",
       "       [ 0.66875485,  0.59349705],\n",
       "       [ 0.44663835, -1.20995865],\n",
       "       [ 0.66875485,  0.59349705]])"
      ]
     },
     "execution_count": 46,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "np.tile(arr, (2, 1))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 47,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "array([[ 0.44663835, -1.20995865,  0.44663835, -1.20995865],\n",
       "       [ 0.66875485,  0.59349705,  0.66875485,  0.59349705],\n",
       "       [ 0.44663835, -1.20995865,  0.44663835, -1.20995865],\n",
       "       [ 0.66875485,  0.59349705,  0.66875485,  0.59349705],\n",
       "       [ 0.44663835, -1.20995865,  0.44663835, -1.20995865],\n",
       "       [ 0.66875485,  0.59349705,  0.66875485,  0.59349705]])"
      ]
     },
     "execution_count": 47,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "np.tile(arr, (3, 2))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 花式索引等效项：take和put\n",
    "\n",
    "您可能从第4章中回忆过，获取和设置数组子集的一种方法是使用整数数组进行华丽索引："
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 48,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "array([700, 100, 200, 600])"
      ]
     },
     "execution_count": 48,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "arr = np.arange(10) * 100\n",
    "\n",
    "inds = [7, 1, 2, 6]\n",
    "arr[inds]"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "在仅在单个轴上进行选择的特殊情况下，还有其他ndarray方法很有用："
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 49,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "array([700, 100, 200, 600])"
      ]
     },
     "execution_count": 49,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "arr.take(inds)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 50,
   "metadata": {},
   "outputs": [],
   "source": [
    "arr.put(inds, 42)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 51,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "array([  0,  42,  42, 300, 400, 500,  42,  42, 800, 900])"
      ]
     },
     "execution_count": 51,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "arr"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 52,
   "metadata": {},
   "outputs": [],
   "source": [
    "arr.put(inds, [40, 41, 42, 43])"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 53,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "array([  0,  41,  42, 300, 400, 500,  43,  40, 800, 900])"
      ]
     },
     "execution_count": 53,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "arr"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "要沿其他轴使用汇整，可以传递axis关键字："
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 54,
   "metadata": {},
   "outputs": [],
   "source": [
    "inds = [2, 0, 2, 1]\n",
    "arr = np.random.randn(2, 4)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 55,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "array([[ 0.47172788, -1.01467909, -0.62255594, -1.15780264],\n",
       "       [ 0.39554734, -0.84808447,  0.32981324,  0.28452574]])"
      ]
     },
     "execution_count": 55,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "arr"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 56,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "array([[-0.62255594,  0.47172788, -0.62255594, -1.01467909],\n",
       "       [ 0.32981324,  0.39554734,  0.32981324, -0.84808447]])"
      ]
     },
     "execution_count": 56,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "arr.take(inds, axis=1)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "put不接受轴参数，而是索引到数组的展平（一维，C阶）版本。因此，当您需要使用其他轴上的索引数组设置元素时，通常最容易使用花式索引。\n",
    "\n",
    "## A.3 广播\n",
    "\n",
    "广播描述了算术如何在不同形状的数组之间工作。它可能是一项强大的功能，但即使对于有经验的用户，也可能引起混乱。最简单的广播示例是将标量值与数组组合在一起时发生的："
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 57,
   "metadata": {},
   "outputs": [],
   "source": [
    "arr = np.arange(5)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 58,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "array([0, 1, 2, 3, 4])"
      ]
     },
     "execution_count": 58,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "arr"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 59,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "array([ 0,  4,  8, 12, 16])"
      ]
     },
     "execution_count": 59,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "arr * 4"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "在这里，我们说标量值4已在乘法运算中广播到所有其他元素。\n",
    "\n",
    "例如，我们可以通过减去列均值来减少数组的每一列。在这种情况下，这非常简单："
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 60,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "array([ 0.29631735, -0.64023109, -0.32615285])"
      ]
     },
     "execution_count": 60,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "arr = np.random.randn(4, 3)\n",
    "\n",
    "arr.mean(0)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 61,
   "metadata": {},
   "outputs": [],
   "source": [
    "demeaned = arr - arr.mean(0)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 62,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "array([[-0.3871364 , -0.93325951,  0.46995205],\n",
       "       [ 0.17072873,  1.12699773, -0.12339537],\n",
       "       [ 0.44921085, -1.05789749,  0.64636361],\n",
       "       [-0.23280318,  0.86415928, -0.99292029]])"
      ]
     },
     "execution_count": 62,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "demeaned"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 63,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "array([-1.38777878e-17,  0.00000000e+00, -2.77555756e-17])"
      ]
     },
     "execution_count": 63,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "demeaned.mean(0)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "有关此操作的说明，请参见图A-4。将行取消为广播操作需要多加注意。幸运的是，只要遵循规则，就有可能在数组的任何维度上广播潜在的较低维度值（例如，从二维数组的每一列中减去行均值）。\n",
    "\n",
    "![app4](images/app.4.png)\n",
    "\n",
    "这使我们能够：\n",
    "\n",
    "> 广播规则\n",
    "> 如果对于每个尾随尺寸（即，从末端开始），轴长度匹配，或者如果两个长度中的任何一个为1，则两个阵列都可以进行广播。然后在丢失或长度为1的尺寸上执行广播。\n",
    "\n",
    "即使是经验丰富的NumPy用户，当我考虑广播规则时，我也经常不得不停下来画一个图。考虑上一个示例，并假设我们希望从每行中减去平均值。由于arr.mean（0）的长度为3，因此它兼容跨轴0广播，因为arr中的尾随尺寸为3，因此匹配。根据规则，要在轴1上减去（即从每行中减去行均值），较小的数组必须具有形状（4，1）："
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 64,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "array([[-0.09081904, -1.5734906 ,  0.1437992 ],\n",
       "       [ 0.46704608,  0.48676664, -0.44954823],\n",
       "       [ 0.7455282 , -1.69812858,  0.32021075],\n",
       "       [ 0.06351417,  0.22392819, -1.31907314]])"
      ]
     },
     "execution_count": 64,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "arr"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 65,
   "metadata": {},
   "outputs": [],
   "source": [
    "row_means = arr.mean(1)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 66,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "(4,)"
      ]
     },
     "execution_count": 66,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "row_means.shape"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 67,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "array([[-0.50683681],\n",
       "       [ 0.16808816],\n",
       "       [-0.21079654],\n",
       "       [-0.34387693]])"
      ]
     },
     "execution_count": 67,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "row_means.reshape((4, 1))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 68,
   "metadata": {},
   "outputs": [],
   "source": [
    "demeaned = arr - row_means.reshape((4, 1))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 69,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "array([1.11022302e-16, 0.00000000e+00, 0.00000000e+00, 3.70074342e-17])"
      ]
     },
     "execution_count": 69,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "demeaned.mean(1)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "有关此操作的说明，请参见图A-5。\n",
    "\n",
    "![app5](images/app.5.png)\n",
    "\n",
    "参见图A-6，这是另一个说明，这一次将一个二维数组添加到一个跨轴0的三维数组中。\n",
    "\n",
    "![app6](images/app.6.png)\n",
    "\n",
    "### 通过其他轴广播\n",
    "\n",
    "使用高维数组进行广播似乎更加令人费解，但这实际上是遵循规则的问题。如果不这样做，则会出现如下错误："
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 70,
   "metadata": {},
   "outputs": [
    {
     "ename": "ValueError",
     "evalue": "operands could not be broadcast together with shapes (4,3) (4,) ",
     "output_type": "error",
     "traceback": [
      "\u001b[0;31m---------------------------------------------------------------------------\u001b[0m",
      "\u001b[0;31mValueError\u001b[0m                                Traceback (most recent call last)",
      "\u001b[0;32m<ipython-input-70-8b8ada26fac0>\u001b[0m in \u001b[0;36m<module>\u001b[0;34m\u001b[0m\n\u001b[0;32m----> 1\u001b[0;31m \u001b[0marr\u001b[0m \u001b[0;34m-\u001b[0m \u001b[0marr\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mmean\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0;36m1\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0m",
      "\u001b[0;31mValueError\u001b[0m: operands could not be broadcast together with shapes (4,3) (4,) "
     ]
    }
   ],
   "source": [
    "arr - arr.mean(1)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "想要对跨轴（而不是轴0）的较低维度数组执行算术运算是很常见的。根据广播规则，较小数组中的“广播维度”必须为1。在此处显示的行贬低示例中，这意味着将行重塑为形状（4，1）而不是（4，）："
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 71,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "array([[ 0.41601777, -1.06665378,  0.65063601],\n",
       "       [ 0.29895792,  0.31867847, -0.61763639],\n",
       "       [ 0.95632474, -1.48733204,  0.5310073 ],\n",
       "       [ 0.4073911 ,  0.56780512, -0.97519621]])"
      ]
     },
     "execution_count": 71,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "arr - arr.mean(1).reshape((4, 1))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "在三维情况下，在三个维度中的任何一个上进行广播仅是将数据重塑为形状兼容的问题。图A-7很好地可视化了在三维阵列的每个轴上广播所需的形状。\n",
    "\n",
    "![app7](images/app.7.png)\n",
    "\n",
    "因此，一个常见的问题是需要专门为广播目的添加一个长度为1的新轴。使用重塑是一种选择，但是插入轴需要构造一个指示新形状的元组。这通常是一个乏味的练习。因此，NumPy数组为通过索引插入新轴提供了一种特殊的语法。 我们使用特殊的np.newaxis属性以及“ full”切片来插入新轴："
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 72,
   "metadata": {},
   "outputs": [],
   "source": [
    "arr = np.zeros((4, 4))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 73,
   "metadata": {},
   "outputs": [],
   "source": [
    "arr_3d = arr[:, np.newaxis, :]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 74,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "(4, 1, 4)"
      ]
     },
     "execution_count": 74,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "arr_3d.shape"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 75,
   "metadata": {},
   "outputs": [],
   "source": [
    "arr_1d = np.random.normal(size=3)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 76,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "array([[-0.6436621 ],\n",
       "       [ 0.1879725 ],\n",
       "       [-0.56895891]])"
      ]
     },
     "execution_count": 76,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "arr_1d[:, np.newaxis]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 77,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "array([[-0.6436621 ,  0.1879725 , -0.56895891]])"
      ]
     },
     "execution_count": 77,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "arr_1d[np.newaxis, :]"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "因此，如果我们有一个三维数组并想降低轴2的强度，则需要编写："
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 78,
   "metadata": {},
   "outputs": [],
   "source": [
    "arr = np.random.randn(3, 4, 5)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 79,
   "metadata": {},
   "outputs": [],
   "source": [
    "depth_means = arr.mean(2)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 80,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "array([[ 0.47335151, -0.13097157, -0.45077735,  0.26695798],\n",
       "       [ 0.50734862,  0.25599043, -0.20621867, -0.13400995],\n",
       "       [-0.17265517, -0.61267556,  0.06057995, -0.77499787]])"
      ]
     },
     "execution_count": 80,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "depth_means"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 81,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "(3, 4)"
      ]
     },
     "execution_count": 81,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "depth_means.shape"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 82,
   "metadata": {},
   "outputs": [],
   "source": [
    "demeaned = arr - depth_means[:, :, np.newaxis]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 83,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "array([[ 2.22044605e-17,  1.11022302e-17, -2.22044605e-17,\n",
       "         0.00000000e+00],\n",
       "       [ 8.88178420e-17, -4.44089210e-17,  1.66533454e-17,\n",
       "         0.00000000e+00],\n",
       "       [ 1.11022302e-17, -2.22044605e-17,  0.00000000e+00,\n",
       "        -2.22044605e-17]])"
      ]
     },
     "execution_count": 83,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "demeaned.mean(2)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "您可能想知道是否有一种方法可以在不牺牲性能的情况下在轴上进行通用归位。有，但是需要一些索引体操："
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 84,
   "metadata": {},
   "outputs": [],
   "source": [
    "def demean_axis(arr, axis=0):\n",
    "    means = arr.mean(axis)\n",
    "    \n",
    "    # This generalize things like [:, :, np.newaxis] to N dimensions\n",
    "    indexer = [slice(None)] * arr.ndim\n",
    "    indexer[axis] = np.newaxis\n",
    "    return arr - means[tuple(indexer)]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 85,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "array([[-3.70074342e-17,  2.22044605e-16,  1.85037171e-17,\n",
       "        -9.25185854e-18,  4.62592927e-17],\n",
       "       [-3.70074342e-17,  7.40148683e-17,  9.25185854e-18,\n",
       "         0.00000000e+00, -1.85037171e-17],\n",
       "       [ 3.70074342e-17,  0.00000000e+00,  1.11022302e-16,\n",
       "         1.85037171e-17, -1.85037171e-17],\n",
       "       [ 0.00000000e+00,  0.00000000e+00,  0.00000000e+00,\n",
       "        -7.40148683e-17,  3.70074342e-17]])"
      ]
     },
     "execution_count": 85,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "demean_axis(arr).mean(0)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 通过广播设置数组值\n",
    "\n",
    "相同的控制算术运算的广播规则也适用于通过数组索引设置值。在简单的情况下，我们可以执行以下操作："
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 86,
   "metadata": {},
   "outputs": [],
   "source": [
    "arr = np.zeros((4, 3))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 87,
   "metadata": {},
   "outputs": [],
   "source": [
    "arr[:] = 5"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 88,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "array([[5., 5., 5.],\n",
       "       [5., 5., 5.],\n",
       "       [5., 5., 5.],\n",
       "       [5., 5., 5.]])"
      ]
     },
     "execution_count": 88,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "arr"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "但是，如果我们想将一维值数组设置为数组的列，那么只要形状兼容，我们就可以这样做："
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 89,
   "metadata": {},
   "outputs": [],
   "source": [
    "col = np.array([1.28, -0.42, 0.44, 1.6])"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 90,
   "metadata": {},
   "outputs": [],
   "source": [
    "arr[:] = col[:, np.newaxis]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 91,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "array([[ 1.28,  1.28,  1.28],\n",
       "       [-0.42, -0.42, -0.42],\n",
       "       [ 0.44,  0.44,  0.44],\n",
       "       [ 1.6 ,  1.6 ,  1.6 ]])"
      ]
     },
     "execution_count": 91,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "arr"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 92,
   "metadata": {},
   "outputs": [],
   "source": [
    "arr[:2] = [[-1.37], [0.509]]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 93,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "array([[-1.37 , -1.37 , -1.37 ],\n",
       "       [ 0.509,  0.509,  0.509],\n",
       "       [ 0.44 ,  0.44 ,  0.44 ],\n",
       "       [ 1.6  ,  1.6  ,  1.6  ]])"
      ]
     },
     "execution_count": 93,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "arr"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## A.4 高级ufunc用法\n",
    "\n",
    "虽然许多NumPy用户将仅使用通用函数提供的快速的按元素操作，但仍有许多其他功能可以偶尔帮助您编写更简洁的代码而无需循环。\n",
    "\n",
    "### ufunc实例方法\n",
    "\n",
    "NumPy的每个二进制ufunc都有执行某些特殊矢量化操作的特殊方法。表A-2中总结了这些内容，但我将给出一些具体示例来说明它们的工作方式。\n",
    "\n",
    "reduce执行单个二进制数组，并通过执行一系列二进制运算来汇总其值（可选地沿一个轴）。例如，对数组中的元素求和的另一种方法是使用np.add.reduce："
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 94,
   "metadata": {},
   "outputs": [],
   "source": [
    "arr = np.arange(10)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 95,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "45"
      ]
     },
     "execution_count": 95,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "np.add.reduce(arr)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 96,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "45"
      ]
     },
     "execution_count": 96,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "arr.sum()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "起始值（0用于添加）取决于ufunc。如果通过了轴，则将沿该轴进行缩小。使您能够以简洁的方式回答某些类型的问题。举一个简单的例子，我们可以使用np.logical_and来检查数组每一行中的值是否已排序："
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 97,
   "metadata": {},
   "outputs": [],
   "source": [
    "np.random.seed(12346) # for reproducibity"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 98,
   "metadata": {},
   "outputs": [],
   "source": [
    "arr = np.random.randn(5, 5)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 99,
   "metadata": {},
   "outputs": [],
   "source": [
    "arr[::2].sort(1) # sort a few rows"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 100,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "array([[ True,  True,  True,  True],\n",
       "       [False,  True, False, False],\n",
       "       [ True,  True,  True,  True],\n",
       "       [ True, False,  True,  True],\n",
       "       [ True,  True,  True,  True]])"
      ]
     },
     "execution_count": 100,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "arr[:, :-1] < arr[:, 1:]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 101,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "array([ True, False,  True, False,  True])"
      ]
     },
     "execution_count": 101,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "np.logical_and.reduce(arr[:, :-1] < arr[:, 1:], axis=1)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "请注意，logical_and.reduce等效于all方法。\n",
    "\n",
    "累积与reduce有关，例如与累积有关。它产生具有中间“累加”值的相同大小的数组："
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 102,
   "metadata": {},
   "outputs": [],
   "source": [
    "arr = np.arange(15).reshape((3, 5))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 103,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "array([[ 0,  1,  3,  6, 10],\n",
       "       [ 5, 11, 18, 26, 35],\n",
       "       [10, 21, 33, 46, 60]])"
      ]
     },
     "execution_count": 103,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "np.add.accumulate(arr, axis=1)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "outer执行两个数组之间的成对叉积："
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 104,
   "metadata": {},
   "outputs": [],
   "source": [
    "arr = np.arange(3).repeat([1, 2, 2])"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 105,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "array([0, 1, 1, 2, 2])"
      ]
     },
     "execution_count": 105,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "arr"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 106,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "array([[0, 0, 0, 0, 0],\n",
       "       [0, 1, 2, 3, 4],\n",
       "       [0, 1, 2, 3, 4],\n",
       "       [0, 2, 4, 6, 8],\n",
       "       [0, 2, 4, 6, 8]])"
      ]
     },
     "execution_count": 106,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "np.multiply.outer(arr, np.arange(5))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "外部的输出将具有一个维度，该维度是输入的维度之和："
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 107,
   "metadata": {},
   "outputs": [],
   "source": [
    "x, y = np.random.randn(3, 4), np.random.randn(5)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 108,
   "metadata": {},
   "outputs": [],
   "source": [
    "result = np.subtract.outer(x, y)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 109,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "(3, 4, 5)"
      ]
     },
     "execution_count": 109,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "result.shape"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "最后一种方法reduceat执行“局部归约”，本质上是对数组进行分组操作，其中将数组的切片聚合在一起。它接受一系列“ bin边”，这些边指示如何拆分和汇总值："
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 110,
   "metadata": {},
   "outputs": [],
   "source": [
    "arr = np.arange(10)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 111,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "array([10, 18, 17])"
      ]
     },
     "execution_count": 111,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "np.add.reduceat(arr, [0, 5, 8])"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "结果是对arr [0：5]，arr [5：8]和arr [8：]执行的减少量（此处为总和）。与其他方法一样，您可以传递轴参数："
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 112,
   "metadata": {},
   "outputs": [],
   "source": [
    "arr = np.multiply.outer(np.arange(4), np.arange(5))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 113,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "array([[ 0,  0,  0,  0,  0],\n",
       "       [ 0,  1,  2,  3,  4],\n",
       "       [ 0,  2,  4,  6,  8],\n",
       "       [ 0,  3,  6,  9, 12]])"
      ]
     },
     "execution_count": 113,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "arr"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 114,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "array([[ 0,  0,  0],\n",
       "       [ 1,  5,  4],\n",
       "       [ 2, 10,  8],\n",
       "       [ 3, 15, 12]])"
      ]
     },
     "execution_count": 114,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "np.add.reduceat(arr, [0, 2, 4], axis=1)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "有关ufunc方法的部分列表，请参见表A-2。\n",
    "\n",
    "### 用Python编写新的ufunc\n",
    "\n",
    "有许多用于创建自己的NumPy ufuncs的工具。最通用的方法是使用NumPy C API，但这超出了本书的范围。在本节中，我们将介绍纯Python ufunc。\n",
    "\n",
    "numpy.frompyfunc接受Python函数以及输入和输出数量的规范。例如，将一个简单的函数逐个元素地添加为："
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 115,
   "metadata": {},
   "outputs": [],
   "source": [
    "def add_elements(x, y):\n",
    "    return x + y"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 116,
   "metadata": {},
   "outputs": [],
   "source": [
    "add_them = np.frompyfunc(add_elements, 2, 1)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 117,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "array([0, 2, 4, 6, 8, 10, 12, 14], dtype=object)"
      ]
     },
     "execution_count": 117,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "add_them(np.arange(8), np.arange(8))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "使用frompyfunc创建的函数总是返回Python对象数组，这可能很不方便。幸运的是，还有一个替代的（但功能稍差一些）函数numpy.vectorize，它允许您指定输出类型："
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 118,
   "metadata": {},
   "outputs": [],
   "source": [
    "add_them = np.vectorize(add_elements, otypes=[np.float64])"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 119,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "array([ 0.,  2.,  4.,  6.,  8., 10., 12., 14.])"
      ]
     },
     "execution_count": 119,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "add_them(np.arange(8), np.arange(8))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "这些函数提供了一种创建类似ufunc的函数的方法，但是它们非常慢，因为它们需要Python函数调用来计算每个元素，这比NumPy基于C的ufunc循环要慢得多："
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 120,
   "metadata": {},
   "outputs": [],
   "source": [
    "arr = np.random.randn(10000)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 121,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "2.09 ms ± 543 µs per loop (mean ± std. dev. of 7 runs, 100 loops each)\n"
     ]
    }
   ],
   "source": [
    "%timeit add_them(arr, arr)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 122,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "6.66 µs ± 911 ns per loop (mean ± std. dev. of 7 runs, 100000 loops each)\n"
     ]
    }
   ],
   "source": [
    "%timeit np.add(arr, arr)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "在本章的后面，我们将展示如何使用Numba项目在Python中创建快速ufunc。\n",
    "\n",
    "### A.5 结构化和记录阵列\n",
    "\n",
    "到目前为止，您可能已经注意到ndarray是同构数据容器。也就是说，它代表一个内存块，其中每个元素占用相同数量的字节（由dtype确定）。从表面上看，这似乎不允许您表示异构或类似表格的数据。结构化数组是一个ndarray，其中每个元素都可以视为代表C中的结构（因此称为“结构化”名称）或SQL表中具有多个命名字段的行："
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 123,
   "metadata": {},
   "outputs": [],
   "source": [
    "dtype = [('x', np.float64), ('y', np.int32)]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 124,
   "metadata": {},
   "outputs": [],
   "source": [
    "sarr = np.array([(1.5, 6), (np.pi, -2)], dtype=(dtype))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 125,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "array([(1.5       ,  6), (3.14159265, -2)],\n",
       "      dtype=[('x', '<f8'), ('y', '<i4')])"
      ]
     },
     "execution_count": 125,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "sarr"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "有几种指定结构化dtype的方法（请参见在线NumPy文档）。一种典型的方法是使用（field_name，field_data_type）的元组列表。现在，数组的元素是类似元组的对象，其元素可以像字典一样进行访问："
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 126,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "(1.5, 6)"
      ]
     },
     "execution_count": 126,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "sarr[0]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 127,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "6"
      ]
     },
     "execution_count": 127,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "sarr[0]['y']"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "字段名称存储在dtype.names属性中。当您访问结构化数组上的字段时，将返回数据的跨步视图，因此不会复制任何内容："
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 128,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "array([1.5       , 3.14159265])"
      ]
     },
     "execution_count": 128,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "sarr['x']"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 嵌套dtype和多维字段\n",
    "\n",
    "当指定结构化dtype时，您还可以传递形状（作为int或tuple）："
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 129,
   "metadata": {},
   "outputs": [],
   "source": [
    "dtype = [('x', np.int64, 3), ('y', np.int32)]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 130,
   "metadata": {},
   "outputs": [],
   "source": [
    "arr = np.zeros(4, dtype=dtype)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 131,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "array([([0, 0, 0], 0), ([0, 0, 0], 0), ([0, 0, 0], 0), ([0, 0, 0], 0)],\n",
       "      dtype=[('x', '<i8', (3,)), ('y', '<i4')])"
      ]
     },
     "execution_count": 131,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "arr"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "在这种情况下，x字段现在指向每个记录的长度为3的数组："
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 132,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "array([0, 0, 0])"
      ]
     },
     "execution_count": 132,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "arr[0]['x']"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "方便地，访问arr ['x']然后返回一个二维数组，而不是前面的示例中的一维数组："
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 133,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "array([[0, 0, 0],\n",
       "       [0, 0, 0],\n",
       "       [0, 0, 0],\n",
       "       [0, 0, 0]])"
      ]
     },
     "execution_count": 133,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "arr['x']"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "这使您可以将更复杂的嵌套结构表示为数组中的单个内存块。您还可以嵌套dtype以使结构更复杂。这是一个例子："
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 134,
   "metadata": {},
   "outputs": [],
   "source": [
    "dtype = [('x', [('a', 'f8'), ('b', 'f4')]), ('y', np.int32)]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 135,
   "metadata": {},
   "outputs": [],
   "source": [
    "data = np.array([((1, 2), 5), ((3, 4), 6)], dtype=dtype)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 136,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "array([(1., 2.), (3., 4.)], dtype=[('a', '<f8'), ('b', '<f4')])"
      ]
     },
     "execution_count": 136,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "data['x']"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 137,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "array([5, 6], dtype=int32)"
      ]
     },
     "execution_count": 137,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "data['y']"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 138,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "array([1., 3.])"
      ]
     },
     "execution_count": 138,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "data['x']['a']"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "pandas DataFrame不直接支持此功能，尽管它类似于分层索引。\n",
    "\n",
    "### 为什么要使用结构化数组？\n",
    "\n",
    "与pandas DataFrame相比，NumPy结构化数组是一个相对较低级别的工具。它们提供了一种将内存块解释为具有任意复杂嵌套列的表格结构的方法。由于数组中的每个元素在内存中均以固定的字节数表示，因此结构化数组提供了一种非常快速有效的方式来往磁盘（包括内存映射）写入数据，在网络上传输数据以及其他类似用途。\n",
    "\n",
    "作为结构化数组的另一种常用用法，将数据文件作为固定长度的记录字节流写入是用C和C ++代码序列化数据的一种常用方法，这在工业遗留系统中很常见。只要知道文件的格式（每个记录的大小以及每个元素的顺序，字节大小和数据类型），就可以使用np.fromfile将数据读入内存。像这样的专门用途不在本书的讨论范围之内，但是值得一提的是这样的事情是可能的。\n",
    "\n",
    "## A.6 有关排序的更多信息\n",
    "\n",
    "像Python的内置列表一样，ndarray排序实例方法是就地排序，这意味着无需重新生成数组即可重新排列数组内容："
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 139,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "array([-1.08199644,  0.37588273,  0.80139193,  1.13969136,  1.28881614,\n",
       "        1.84126094])"
      ]
     },
     "execution_count": 139,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "arr = np.random.randn(6)\n",
    "arr.sort()\n",
    "arr"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "就地对数组进行排序时，请记住，如果该数组是另一个ndarray上的视图，则将修改原始数组："
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 140,
   "metadata": {},
   "outputs": [],
   "source": [
    "arr = np.random.randn(3, 5)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 141,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "array([[-0.33176812, -1.47108206,  0.87050269, -0.08468875, -1.13286962],\n",
       "       [-1.01114869, -0.34357617,  2.17140268,  0.12337075, -0.01893118],\n",
       "       [ 0.17731791,  0.7423957 ,  0.85475634,  1.03797268, -0.32899594]])"
      ]
     },
     "execution_count": 141,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "arr"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 142,
   "metadata": {},
   "outputs": [],
   "source": [
    "arr[:, 0].sort()  # Sort first column values in-place"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 143,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "array([[-1.01114869, -1.47108206,  0.87050269, -0.08468875, -1.13286962],\n",
       "       [-0.33176812, -0.34357617,  2.17140268,  0.12337075, -0.01893118],\n",
       "       [ 0.17731791,  0.7423957 ,  0.85475634,  1.03797268, -0.32899594]])"
      ]
     },
     "execution_count": 143,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "arr"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "另一方面，numpy.sort创建一个新的，已排序的数组副本。否则，它接受与ndarray.sort相同的参数（例如kind）："
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 144,
   "metadata": {},
   "outputs": [],
   "source": [
    "arr = np.random.randn(5)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 145,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "array([-1.11807759, -0.24152521, -2.0051193 ,  0.73788753, -1.06137462])"
      ]
     },
     "execution_count": 145,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "arr"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 146,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "array([-2.0051193 , -1.11807759, -1.06137462, -0.24152521,  0.73788753])"
      ]
     },
     "execution_count": 146,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "np.sort(arr)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 147,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "array([-1.11807759, -0.24152521, -2.0051193 ,  0.73788753, -1.06137462])"
      ]
     },
     "execution_count": 147,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "arr"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 148,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "array([-2.0051193 , -1.11807759, -1.06137462, -0.24152521,  0.73788753])"
      ]
     },
     "execution_count": 148,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "np.sort(arr)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 149,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "array([-1.11807759, -0.24152521, -2.0051193 ,  0.73788753, -1.06137462])"
      ]
     },
     "execution_count": 149,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "arr"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "所有这些排序方法都带有一个axis参数，用于沿传递的轴独立地对数据段进行排序："
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 150,
   "metadata": {},
   "outputs": [],
   "source": [
    "arr = np.random.randn(3, 5)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 151,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "array([[ 0.59545348, -0.26822958,  1.33885804, -0.18715572,  0.91108374],\n",
       "       [-0.32150045,  1.00543901, -0.51683937,  1.19251887, -0.19893404],\n",
       "       [ 0.39691349, -1.76381537,  0.60709023, -0.22215536, -0.21707838]])"
      ]
     },
     "execution_count": 151,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "arr"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 152,
   "metadata": {},
   "outputs": [],
   "source": [
    "arr.sort(axis=1)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 153,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "array([[-0.26822958, -0.18715572,  0.59545348,  0.91108374,  1.33885804],\n",
       "       [-0.51683937, -0.32150045, -0.19893404,  1.00543901,  1.19251887],\n",
       "       [-1.76381537, -0.22215536, -0.21707838,  0.39691349,  0.60709023]])"
      ]
     },
     "execution_count": 153,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "arr"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "您可能会注意到，排序方法都没有一个选项可以按降序排序。实际上，这是一个问题，因为数组切片会产生视图，因此不会产生副本或需要任何计算工作。 许多Python用户熟悉“技巧”，即对于列表值，values [::-1]以相反的顺序返回列表。 ndarrays也是如此："
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 154,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "array([[ 1.33885804,  0.91108374,  0.59545348, -0.18715572, -0.26822958],\n",
       "       [ 1.19251887,  1.00543901, -0.19893404, -0.32150045, -0.51683937],\n",
       "       [ 0.60709023,  0.39691349, -0.21707838, -0.22215536, -1.76381537]])"
      ]
     },
     "execution_count": 154,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "arr[:, ::-1]"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 间接排序：argsort和lexsort\n",
    "\n",
    "在数据分析中，您可能需要通过一个或多个键对数据集进行重新排序。例如，关于某些学生的数据表可能需要按姓氏，然后按名字排序。这是一个间接排序的示例，如果您已经阅读了与熊猫相关的章节，那么您已经看到了许多更高级别的示例。给定一个或多个键（一个值数组或多个值数组），您希望获得一个整数索引数组（我通常将其称为索引器），告诉您如何将数据重新排序为排序后的顺序。两种方法是argsort和numpy.lexsort。举个例子："
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 155,
   "metadata": {},
   "outputs": [],
   "source": [
    "values = np.array([5, 0, 1, 3, 2])"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 156,
   "metadata": {},
   "outputs": [],
   "source": [
    "indexer = values.argsort()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 157,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "array([1, 2, 4, 3, 0])"
      ]
     },
     "execution_count": 157,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "indexer"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 158,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "array([0, 1, 2, 3, 5])"
      ]
     },
     "execution_count": 158,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "values[indexer]"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "作为一个更复杂的示例，此代码按其第一行对二维数组进行重新排序："
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 159,
   "metadata": {},
   "outputs": [],
   "source": [
    "arr = np.random.randn(3, 5)\n",
    "\n",
    "arr[0] = values"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 160,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "array([[ 5.        ,  0.        ,  1.        ,  3.        ,  2.        ],\n",
       "       [-0.36360302, -0.13775933,  2.17773731, -0.47280687,  0.8356152 ],\n",
       "       [-0.20885016,  0.23159352,  0.72798172, -1.3918432 ,  1.99558262]])"
      ]
     },
     "execution_count": 160,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "arr"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 161,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "array([[ 0.        ,  1.        ,  2.        ,  3.        ,  5.        ],\n",
       "       [-0.13775933,  2.17773731,  0.8356152 , -0.47280687, -0.36360302],\n",
       "       [ 0.23159352,  0.72798172,  1.99558262, -1.3918432 , -0.20885016]])"
      ]
     },
     "execution_count": 161,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "arr[:, arr[0].argsort()]"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "lexsort与argsort相似，但是它对多个键数组执行间接字典排序。假设我们要对一些由名字和姓氏标识的数据进行排序："
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 162,
   "metadata": {},
   "outputs": [],
   "source": [
    "first_name = np.array(['Bob', 'Jane', 'Steve', 'Bill', 'Barbara'])"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 163,
   "metadata": {},
   "outputs": [],
   "source": [
    "last_name = np.array(['Jones', 'Arnold', 'Arnold', 'Jones', 'Walters'])"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 164,
   "metadata": {},
   "outputs": [],
   "source": [
    "sorter = np.lexsort([first_name, last_name])"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 165,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "array([1, 2, 3, 0, 4])"
      ]
     },
     "execution_count": 165,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "sorter"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 166,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "[('Arnold', 'Jane'),\n",
       " ('Arnold', 'Steve'),\n",
       " ('Jones', 'Bill'),\n",
       " ('Jones', 'Bob'),\n",
       " ('Walters', 'Barbara')]"
      ]
     },
     "execution_count": 166,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "list(zip(last_name[sorter], first_name[sorter]))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "lexsort第一次使用时可能会有些混乱，因为使用键对数据进行排序的顺序是从最后一个传递的数组开始的。在此，last_name在first_name之前使用。\n",
    "\n",
    "> Series和DataFrame的sort_values方法之类的pandas方法是通过这些函数的变体实现的（还必须考虑缺失值）。\n",
    "\n",
    "### 替代排序算法\n",
    "\n",
    "稳定的排序算法可保留相等元素的相对位置。这在相对排序有意义的间接排序中尤其重要："
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 167,
   "metadata": {},
   "outputs": [],
   "source": [
    "values = np.array(['2:first', '2:second', '1:first', '1:second', '1:third'])"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 168,
   "metadata": {},
   "outputs": [],
   "source": [
    "key = np.array([2, 2, 1, 1, 1])"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 169,
   "metadata": {},
   "outputs": [],
   "source": [
    "indexer = key.argsort(kind='mergesort')"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 170,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "array([2, 3, 4, 0, 1])"
      ]
     },
     "execution_count": 170,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "indexer"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 171,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "array(['1:first', '1:second', '1:third', '2:first', '2:second'],\n",
       "      dtype='<U8')"
      ]
     },
     "execution_count": 171,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "values.take(indexer)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "唯一可用的稳定排序是mergesort，它可以保证O（n log n）性能（针对复杂性增强），但其性能平均比默认的quicksort方法差。有关可用方法及其相对性能（和性能保证）的摘要，请参见表A-3。大多数用户都不必考虑这一点，但是知道它在那里很有用。\n",
    "\n",
    "### 部分排序数组\n",
    "\n",
    "排序的目标之一可以是确定数组中的最大或最小元素。NumPy具有优化的方法numpy.partition和np.argpartition，用于围绕第k个最小元素对数组进行分区："
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 172,
   "metadata": {},
   "outputs": [],
   "source": [
    "np.random.seed(12345)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 173,
   "metadata": {},
   "outputs": [],
   "source": [
    "arr = np.random.randn(20)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 174,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "array([-0.20470766,  0.47894334, -0.51943872, -0.5557303 ,  1.96578057,\n",
       "        1.39340583,  0.09290788,  0.28174615,  0.76902257,  1.24643474,\n",
       "        1.00718936, -1.29622111,  0.27499163,  0.22891288,  1.35291684,\n",
       "        0.88642934, -2.00163731, -0.37184254,  1.66902531, -0.43856974])"
      ]
     },
     "execution_count": 174,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "arr"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 175,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "array([-2.00163731, -1.29622111, -0.5557303 , -0.51943872, -0.37184254,\n",
       "       -0.43856974, -0.20470766,  0.28174615,  0.76902257,  0.47894334,\n",
       "        1.00718936,  0.09290788,  0.27499163,  0.22891288,  1.35291684,\n",
       "        0.88642934,  1.39340583,  1.96578057,  1.66902531,  1.24643474])"
      ]
     },
     "execution_count": 175,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "np.partition(arr, 3)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "调用partition（arr，3）之后，结果中的前三个元素是没有特定顺序的最小三个值。numpy.argpartition与numpy.arg sort类似，返回将数据重新排列为等效顺序的索引："
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 176,
   "metadata": {},
   "outputs": [],
   "source": [
    "indices = np.argpartition(arr, 3)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 177,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "array([16, 11,  3,  2, 17, 19,  0,  7,  8,  1, 10,  6, 12, 13, 14, 15,  5,\n",
       "        4, 18,  9])"
      ]
     },
     "execution_count": 177,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "indices"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 178,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "array([-2.00163731, -1.29622111, -0.5557303 , -0.51943872, -0.37184254,\n",
       "       -0.43856974, -0.20470766,  0.28174615,  0.76902257,  0.47894334,\n",
       "        1.00718936,  0.09290788,  0.27499163,  0.22891288,  1.35291684,\n",
       "        0.88642934,  1.39340583,  1.96578057,  1.66902531,  1.24643474])"
      ]
     },
     "execution_count": 178,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "arr.take(indices)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### numpy.searchsorted：在排序数组中查找元素\n",
    "\n",
    "searchsorted是一种数组方法，它对已排序的数组执行二进制搜索，并返回该数组中需要插入值以保持排序的位置："
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 179,
   "metadata": {},
   "outputs": [],
   "source": [
    "arr = np.array([0, 1, 7, 12, 15])"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 180,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "3"
      ]
     },
     "execution_count": 180,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "arr.searchsorted(9)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "您还可以传递值数组以返回索引数组："
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 181,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "array([0, 3, 3, 5])"
      ]
     },
     "execution_count": 181,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "arr.searchsorted([0, 8, 11, 16])"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "您可能已经注意到searchsorted为0元素返回了0。这是因为默认行为是在一组相等值的左侧返回索引："
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 182,
   "metadata": {},
   "outputs": [],
   "source": [
    "arr = np.array([0, 0, 0, 1, 1, 1, 1])"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 183,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "array([0, 3])"
      ]
     },
     "execution_count": 183,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "arr.searchsorted([0, 1])"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 184,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "array([3, 7])"
      ]
     },
     "execution_count": 184,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "arr.searchsorted([0, 1], side='right')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "作为searchsorted的另一个应用程序，假设我们有一个0到10,000之间的值数组，以及一个我们想用来对数据进行装箱的单独的“ bucket edge”数组："
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 185,
   "metadata": {},
   "outputs": [],
   "source": [
    "data = np.floor(np.random.uniform(0, 10000, size=50))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 187,
   "metadata": {},
   "outputs": [],
   "source": [
    "bins = np.array([0, 100, 1000, 5000, 10000])"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 188,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "array([9940., 6768., 7908., 1709.,  268., 8003., 9037.,  246., 4917.,\n",
       "       5262., 5963.,  519., 8950., 7282., 8183., 5002., 8101.,  959.,\n",
       "       2189., 2587., 4681., 4593., 7095., 1780., 5314., 1677., 7688.,\n",
       "       9281., 6094., 1501., 4896., 3773., 8486., 9110., 3838., 3154.,\n",
       "       5683., 1878., 1258., 6875., 7996., 5735., 9732., 6340., 8884.,\n",
       "       4954., 3516., 7142., 5039., 2256.])"
      ]
     },
     "execution_count": 188,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "data"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "为了获得每个数据点属于哪个间隔的标签（其中1表示存储桶[0，100]），我们可以简单地使用searchsorted："
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 189,
   "metadata": {},
   "outputs": [],
   "source": [
    "labels = bins.searchsorted(data)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 190,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "array([4, 4, 4, 3, 2, 4, 4, 2, 3, 4, 4, 2, 4, 4, 4, 4, 4, 2, 3, 3, 3, 3,\n",
       "       4, 3, 4, 3, 4, 4, 4, 3, 3, 3, 4, 4, 3, 3, 4, 3, 3, 4, 4, 4, 4, 4,\n",
       "       4, 3, 3, 4, 4, 3])"
      ]
     },
     "execution_count": 190,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "labels"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "结合熊猫的groupby，可用于对数据进行分箱："
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 191,
   "metadata": {},
   "outputs": [],
   "source": [
    "import pandas as pd"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 192,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "2     498.000000\n",
       "3    3064.277778\n",
       "4    7389.035714\n",
       "dtype: float64"
      ]
     },
     "execution_count": 192,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "pd.Series(data).groupby(labels).mean()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## A.7 用Numba编写快速的NumPy函数\n",
    "\n",
    "Numba是一个开源项目，可以使用CPU，GPU或其他硬件为类似NumPy的数据创建快速功能。它使用LLVM项目将Python代码转换为已编译的机器代码。\n",
    "\n",
    "为了介绍Numba，我们考虑一个纯Python函数，该函数使用for循环来计算表达式（x-y）.mean（）："
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 196,
   "metadata": {},
   "outputs": [],
   "source": [
    "import numpy as np\n",
    "\n",
    "def mean_distance(x, y):\n",
    "    nx = len(x)\n",
    "    result = 0.0\n",
    "    count = 0\n",
    "    for i in range(nx):\n",
    "        result += x[i] - y[i]\n",
    "        count += 1\n",
    "    return result / count"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "这个函数很慢："
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 197,
   "metadata": {},
   "outputs": [],
   "source": [
    "x = np.random.randn(10000000)\n",
    "y = np.random.randn(10000000)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 198,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "5.65 s ± 214 ms per loop (mean ± std. dev. of 7 runs, 1 loop each)\n"
     ]
    }
   ],
   "source": [
    "%timeit mean_distance(x, y)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 200,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "34 ms ± 7.75 ms per loop (mean ± std. dev. of 7 runs, 1 loop each)\n"
     ]
    }
   ],
   "source": [
    "%timeit (x - y).mean()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "NumPy版本快100倍以上。我们可以使用numba.jit函数将此函数转换为已编译的Numba函数："
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 202,
   "metadata": {},
   "outputs": [],
   "source": [
    "import numba as nb"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 203,
   "metadata": {},
   "outputs": [],
   "source": [
    "numba_mean_distance = nb.jit(mean_distance)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "我们也可以将其写为装饰器："
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 204,
   "metadata": {},
   "outputs": [],
   "source": [
    "@nb.jit\n",
    "def mean_distance(x, y):\n",
    "    nx = len(x)\n",
    "    result = 0.0\n",
    "    count = 0\n",
    "    for i in range(nx):\n",
    "        result += x[i] - y[i]\n",
    "        count += 1\n",
    "    return result / count"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "结果函数实际上比矢量化NumPy版本快："
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 205,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "21 ms ± 5.82 ms per loop (mean ± std. dev. of 7 runs, 1 loop each)\n"
     ]
    }
   ],
   "source": [
    "%timeit numba_mean_distance(x, y)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Numba无法编译任意Python代码，但它支持纯Python的重要子集，这对编写数值算法最有用。\n",
    "\n",
    "Numba是一个深层库，支持各种硬件，编译模式和用户扩展。 它也能够编译NumPy Python API的实质子集，而无需显式的for循环。 Numba能够识别可编译为机器代码的构造，同时将对CPython API的调用替换为它不知道如何编译的函数。 Numba的jit函数具有nopython = True选项，该选项将允许的代码限制为可在无需任何Python C API调用的情况下编译为LLVM的Python代码。jit（nopython = True）具有较短的别名numba.njit。\n",
    "\n",
    "在前面的示例中，我们可以编写："
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 207,
   "metadata": {},
   "outputs": [],
   "source": [
    "from numba import float64, njit\n",
    "\n",
    "@njit(float64(float64[:], float64[:]))\n",
    "def mean_distance(x, y):\n",
    "    return (x - y).mean()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "我鼓励您阅读Numba的在线文档以了解更多信息。下一部分显示了创建自定义NumPy ufunc对象的示例。\n",
    "\n",
    "### 使用Numba创建自定义numpy.ufunc对象\n",
    "\n",
    "numba.vectorize函数创建已编译的NumPy ufunc，其行为类似于内置ufunc。让我们考虑numpy.add的Python实现："
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 208,
   "metadata": {},
   "outputs": [],
   "source": [
    "from numba import vectorize\n",
    "\n",
    "@vectorize\n",
    "def nb_add(x, y):\n",
    "    return x + y"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "现在我们有："
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 209,
   "metadata": {},
   "outputs": [],
   "source": [
    "x = np.arange(10)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 210,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "array([ 0,  2,  4,  6,  8, 10, 12, 14, 16, 18])"
      ]
     },
     "execution_count": 210,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "nb_add(x, x)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 211,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "array([ 0,  1,  3,  6, 10, 15, 21, 28, 36, 45])"
      ]
     },
     "execution_count": 211,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "nb_add.accumulate(x, 0)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## A.8 高级阵列输入和输出\n",
    "\n",
    "在第4章中，我们熟悉了np.save和np.load，用于将二进制格式的数组存储在磁盘上。对于更复杂的用途，还有许多其他选项可供考虑。特别是，内存映射还具有使您能够处理不适合RAM的数据集的额外好处。\n",
    "\n",
    "### 内存映射文件\n",
    "\n",
    "内存映射文件是一种与磁盘上的二进制数据进行交互的方法，就好像它存储在内存阵列中一样。NumPy实现了一个类似于ndarray的memmap对象，从而可以在不将整个数组读入内存的情况下读取和写入大文件的小片段。另外，memmap具有与内存中数组相同的方法，因此可以替换为期望使用ndarray的许多算法。\n",
    "\n",
    "要创建新的内存映射，请使用函数np.memmap并传递文件路径，dtype，shape和文件模式："
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 212,
   "metadata": {},
   "outputs": [],
   "source": [
    "mmap = np.memmap('mymmap', dtype='float64', mode='w+', shape=(10000, 10000))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 213,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "memmap([[0., 0., 0., ..., 0., 0., 0.],\n",
       "        [0., 0., 0., ..., 0., 0., 0.],\n",
       "        [0., 0., 0., ..., 0., 0., 0.],\n",
       "        ...,\n",
       "        [0., 0., 0., ..., 0., 0., 0.],\n",
       "        [0., 0., 0., ..., 0., 0., 0.],\n",
       "        [0., 0., 0., ..., 0., 0., 0.]])"
      ]
     },
     "execution_count": 213,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "mmap"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "切片memmap会返回磁盘上数据的视图："
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 214,
   "metadata": {},
   "outputs": [],
   "source": [
    "section = mmap[:5]"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "如果为它们分配数据，它将被缓存在内存中（就像Python文件对象一样），但是您可以通过调用flush将其写入磁盘："
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 215,
   "metadata": {},
   "outputs": [],
   "source": [
    "section[:] = np.random.randn(5, 10000)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 216,
   "metadata": {},
   "outputs": [],
   "source": [
    "mmap.flush()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 217,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "memmap([[-0.18245763, -0.98387605, -0.03924531, ..., -2.1232373 ,\n",
       "          0.4699011 ,  1.23045698],\n",
       "        [ 0.27941134,  0.30688684, -0.62014901, ..., -1.99251109,\n",
       "          0.03710872,  0.02795918],\n",
       "        [ 1.27323018,  0.25729472,  1.17426959, ..., -0.57735952,\n",
       "          0.89564726, -1.02518372],\n",
       "        ...,\n",
       "        [ 0.        ,  0.        ,  0.        , ...,  0.        ,\n",
       "          0.        ,  0.        ],\n",
       "        [ 0.        ,  0.        ,  0.        , ...,  0.        ,\n",
       "          0.        ,  0.        ],\n",
       "        [ 0.        ,  0.        ,  0.        , ...,  0.        ,\n",
       "          0.        ,  0.        ]])"
      ]
     },
     "execution_count": 217,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "mmap"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 218,
   "metadata": {},
   "outputs": [],
   "source": [
    "del mmap"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "每当内存映射超出范围并被垃圾回收时，所有更改也将刷新到磁盘。打开现有的内存映射时，您仍然必须指定dtype和shape，因为该文件只是二进制数据块，磁盘上没有元数据："
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 219,
   "metadata": {},
   "outputs": [],
   "source": [
    "mmap = np.memmap('mymmap', dtype='float64', shape=(10000, 10000))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 220,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "memmap([[-0.18245763, -0.98387605, -0.03924531, ..., -2.1232373 ,\n",
       "          0.4699011 ,  1.23045698],\n",
       "        [ 0.27941134,  0.30688684, -0.62014901, ..., -1.99251109,\n",
       "          0.03710872,  0.02795918],\n",
       "        [ 1.27323018,  0.25729472,  1.17426959, ..., -0.57735952,\n",
       "          0.89564726, -1.02518372],\n",
       "        ...,\n",
       "        [ 0.        ,  0.        ,  0.        , ...,  0.        ,\n",
       "          0.        ,  0.        ],\n",
       "        [ 0.        ,  0.        ,  0.        , ...,  0.        ,\n",
       "          0.        ,  0.        ],\n",
       "        [ 0.        ,  0.        ,  0.        , ...,  0.        ,\n",
       "          0.        ,  0.        ]])"
      ]
     },
     "execution_count": 220,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "mmap"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "如上一节所述，内存映射还可以与结构化或嵌套dtypes一起使用。\n",
    "\n",
    "### HDF5和其他阵列存储选项\n",
    "\n",
    "PyTables和h5py是两个Python项目，它们提供了NumPy友好的接口，用于以高效且可压缩的HDF5格式存储阵列数据（HDF代表分层数据格式）。您可以安全地以HDF5格式存储数百GB甚至TB的数据。要了解有关将HDF5与Python结合使用的更多信息，我建议阅读pandas在线文档。\n",
    "\n",
    "## A.9 性能技巧\n",
    "\n",
    "利用NumPy从代码中获得良好的性能通常很简单，因为数组操作通常会替代原本相对缓慢的纯Python循环。以下列表简要总结了要记住的一些事项：\n",
    "\n",
    "- 将Python循环和条件逻辑转换为数组运算和布尔数组运算\n",
    "- 尽可能使用广播\n",
    "- 使用数组视图（切片）以避免复制数据\n",
    "- 使用ufuncs和ufunc方法\n",
    "\n",
    "如果仅用完NumPy提供的功能后仍无法获得所需的性能，请考虑使用C，Fortran或Cython编写代码。我在自己的工作中经常使用Cython，这是通过最少的开发即可获得类似C的性能的简便方法。\n",
    "\n",
    "### 连续存储的重要性\n",
    "\n",
    "尽管本主题的全部内容不在本书的讨论范围之内，但在某些应用程序中，数组的内存布局会显着影响计算速度。这部分是基于与CPU的缓存层次结构有关的性能差异。访问存储器的连续块的操作（例如，对C顺序数组的行求和）通常将是最快的，因为存储器子系统会将适当的存储器块缓冲到超高速L1或L2 CPU高速缓存中。同样，NumPy的C代码库中的某些代码路径已针对连续情况进行了优化，在这种情况下可以避免通用的跨步内存访问。\n",
    "\n",
    "要说数组的内存布局是连续的，则意味着元素按照Fortran（列主）或C（行主）的顺序以它们在数组中出现的顺序存储在内存中。默认情况下，NumPy数组创建为C连续的或只是简单连续的。列主数组（例如C连续数组的转置）因此被称为Fortran连续数组。 这些属性可以通过ndarray的flags属性进行显式检查："
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 221,
   "metadata": {},
   "outputs": [],
   "source": [
    "arr_c = np.ones((1000, 1000), order='C')"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 222,
   "metadata": {},
   "outputs": [],
   "source": [
    "arr_f = np.ones((1000, 1000), order='F')"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 223,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "  C_CONTIGUOUS : True\n",
       "  F_CONTIGUOUS : False\n",
       "  OWNDATA : True\n",
       "  WRITEABLE : True\n",
       "  ALIGNED : True\n",
       "  WRITEBACKIFCOPY : False\n",
       "  UPDATEIFCOPY : False"
      ]
     },
     "execution_count": 223,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "arr_c.flags"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 224,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "  C_CONTIGUOUS : False\n",
       "  F_CONTIGUOUS : True\n",
       "  OWNDATA : True\n",
       "  WRITEABLE : True\n",
       "  ALIGNED : True\n",
       "  WRITEBACKIFCOPY : False\n",
       "  UPDATEIFCOPY : False"
      ]
     },
     "execution_count": 224,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "arr_f.flags"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 225,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "True"
      ]
     },
     "execution_count": 225,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "arr_f.flags.f_contiguous"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "在此示例中，从理论上讲，对arr_c求和这些数组的行应该比对arr_f更快，因为这些行在内存中是连续的。在这里，我确定在IPython中使用％timeit进行检查："
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 226,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "505 µs ± 32.6 µs per loop (mean ± std. dev. of 7 runs, 1000 loops each)\n"
     ]
    }
   ],
   "source": [
    "%timeit arr_c.sum(1)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 227,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "878 µs ± 76.7 µs per loop (mean ± std. dev. of 7 runs, 1000 loops each)\n"
     ]
    }
   ],
   "source": [
    "%timeit arr_f.sum(1)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "当您希望从NumPy中获取更多性能时，通常可以在此花费一些精力。如果您的数组没有所需的内存顺序，则可以使用copy并传递'C'或'F'："
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 228,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "  C_CONTIGUOUS : True\n",
       "  F_CONTIGUOUS : False\n",
       "  OWNDATA : True\n",
       "  WRITEABLE : True\n",
       "  ALIGNED : True\n",
       "  WRITEBACKIFCOPY : False\n",
       "  UPDATEIFCOPY : False"
      ]
     },
     "execution_count": 228,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "arr_f.copy('C').flags"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "在数组上构造视图时，请记住，不能保证结果是连续的："
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 231,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "True"
      ]
     },
     "execution_count": 231,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "arr_c[:50].flags.contiguous"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 233,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "  C_CONTIGUOUS : False\n",
       "  F_CONTIGUOUS : False\n",
       "  OWNDATA : False\n",
       "  WRITEABLE : True\n",
       "  ALIGNED : True\n",
       "  WRITEBACKIFCOPY : False\n",
       "  UPDATEIFCOPY : False"
      ]
     },
     "execution_count": 233,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "arr_c[:, :50].flags"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": []
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.7.4"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 2
}
